[WIP] Adding MultiAgent Utilities #323

AdityaKapoor74 · 2020-09-03T05:38:21Z

No description provided.

lgtm-com · 2020-09-03T06:03:03Z

This pull request introduces 1 alert when merging 0927001 into c3ef4d9 - view on LGTM.com

new alerts:

1 for Variable defined multiple times

Sharad24 · 2020-09-03T06:03:57Z

Looks pretty good.

lgtm-com · 2020-09-03T07:37:43Z

This pull request introduces 2 alerts when merging daa8b2a into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T08:37:37Z

This pull request introduces 2 alerts when merging 44db72e into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T09:22:44Z

This pull request introduces 2 alerts when merging d8cf1a9 into c3ef4d9 - view on LGTM.com

new alerts:

1 for Syntax error
1 for Variable defined multiple times

lgtm-com · 2020-09-03T10:14:30Z

This pull request introduces 5 alerts when merging 1365585 into c3ef4d9 - view on LGTM.com

new alerts:

2 for Syntax error
2 for First argument to super() is not enclosing class
1 for Variable defined multiple times

genrl/core/actor_critic.py

genrl/utils/utils.py

Sharad24 · 2020-09-03T11:18:37Z

genrl/environments/gym_wrapper.py

        self.env.close()
+
+
+class MultiGymWrapper(gym.Wrapper):


Whats the difference between this and a normal Wrapper? Cant you use a TorchWrapper instead of creating this?

This is just a template, I haven't added anything yet

Sharad24

Can you install pre-commit? You can see the instructions in Contributing.md

lgtm-com · 2020-09-03T13:01:20Z

This pull request introduces 5 alerts when merging 6f0563e into c3ef4d9 - view on LGTM.com

new alerts:

2 for Syntax error
2 for First argument to super() is not enclosing class
1 for Variable defined multiple times

lgtm-com · 2020-09-04T06:19:20Z

This pull request introduces 7 alerts when merging 5061abe into 0fe4180 - view on LGTM.com

new alerts:

2 for Mismatch between signature and use of an overriding method
2 for Wrong number of arguments in a class instantiation
1 for First argument to super() is not enclosing class
1 for Variable defined multiple times
1 for Wrong number of arguments in a call

codecov · 2020-09-04T07:10:11Z

Codecov Report

Merging #323 into master will decrease coverage by 3.46%.
The diff coverage is 36.07%.

@@            Coverage Diff             @@
##           master     #323      +/-   ##
==========================================
- Coverage   90.77%   87.31%   -3.47%     
==========================================
  Files          89       89              
  Lines        3772     4027     +255     
==========================================
+ Hits         3424     3516      +92     
- Misses        348      511     +163

Impacted Files	Coverage Δ
genrl/agents/deep/a2c/a2c.py	`93.58% <ø> (ø)`
genrl/agents/deep/ddpg/ddpg.py	`83.05% <0.00%> (-4.45%)`	⬇️
genrl/core/rollout_storage.py	`56.04% <9.21%> (-33.58%)`	⬇️
genrl/environments/suite.py	`73.68% <11.11%> (-19.42%)`	⬇️
genrl/core/buffers.py	`63.72% <12.82%> (-31.52%)`	⬇️
genrl/environments/gym_wrapper.py	`54.21% <27.50%> (-24.86%)`	⬇️
genrl/agents/deep/ppo1/ppo1.py	`97.61% <33.33%> (-2.39%)`	⬇️
genrl/utils/utils.py	`86.62% <78.82%> (-7.63%)`	⬇️

Sharad24 · 2020-09-04T10:49:13Z

.pre-commit-config.yaml


  - repo: https://github.com/timothycrosley/isort
-    rev: 4.3.2
+    rev: 5.5.0


Its very risky to change the versions here because of compatibility. Can you revert these?

When I change that I get some errors. Any other way to rectify it?

Have you changed the isort.cfg file?

genrl/core/actor_critic.py

Sharad24 · 2020-09-04T11:20:43Z

genrl/core/actor_critic.py

+class Critic(MlpValue):
+    def __init__(
+        self,
+        state_dim: spaces.Space,
+        action_dim: spaces.Space,
+        fc_layers: Tuple = (32, 32),
+        val_type: str = "V",
+        **kwargs,
+    ):
+        super(Critic, self).__init__(
+            state_dim, action_dim, fc_layers, val_type, **kwargs
+        )
+
+        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+    def forward(self, state):
+
+        state = self.model(state)
+
+        return state
+
+    def get_value(self, state):
+        # state = torch.FloatTensor(state).to(self.device)
+        value = self.forward(state)
+        return value


Whats different in this Critic? You could just use the MlpValue directly?

I'll add in the changes on the fly. I am just creating a template so that it can be used readily

… shared

genrl/utils/utils.py

genrl/core/actor_critic.py

Shared single ACs

genrl/agents/deep/ppo1/ppo1.py

genrl/core/actor_critic.py

…enrl into multiagentutils

Co-authored-by: Vedant Shah <[email protected]>

hades-rp2010 and others added 7 commits September 1, 2020 15:49

Single actor critic shared params

1d49049

Shared layers for multi ACs

ef4a179

Merge branch 'master' of https://github.com/SforAiDl/genrl

2ecd086

Fix lint errors (1)

53450a8

Fixed tests

274aff9

Changes to dicstrings and classes

38f95f0

adding MultiAgentBuffer

0927001

shared mlp

daa8b2a

adding changes

44db72e

AdityaKapoor74 added 2 commits September 3, 2020 14:14

new mlp for maddpg

4ef8f48

adding environment loader

d8cf1a9

AdityaKapoor74 added 2 commits September 3, 2020 15:10

Adding Actor and Critic classes

8d2cf06

adding new functionalities

1365585

Sharad24 reviewed Sep 3, 2020

View reviewed changes

genrl/core/actor_critic.py Outdated Show resolved Hide resolved

Sharad24 reviewed Sep 3, 2020

View reviewed changes

genrl/utils/utils.py Outdated Show resolved Hide resolved

Sharad24 reviewed Sep 3, 2020

View reviewed changes

AdityaKapoor74 added 2 commits September 3, 2020 18:00

minor changes

5067e42

added return statement to mlp_

6f0563e

rectifying

5061abe

rectifying 2

e6a378c

Sharad24 reviewed Sep 4, 2020

View reviewed changes

hades-rp2010 and others added 6 commits September 4, 2020 22:55

Renaming Multi -> Two and comments

835819e

changing names

793c045

changing names

2635fd5

Merge branch 'master' of https://github.com/AdityaKapoor74/genrl into…

cd87506

… shared

Shared params for single ACs

65b6520

Merge branch 'multiagentutils' into shared

3d01b85

veds12 reviewed Oct 2, 2020

View reviewed changes

genrl/utils/utils.py Outdated Show resolved Hide resolved

veds12 reviewed Oct 2, 2020

View reviewed changes

genrl/core/actor_critic.py Outdated Show resolved Hide resolved

AdityaKapoor74 added 2 commits October 4, 2020 09:35

Merge pull request #1 from hades-rp2010/shared

a62c100

Shared single ACs

Merge branch 'master' into multiagentutils

841ff66

veds12 reviewed Oct 4, 2020

View reviewed changes

genrl/agents/deep/ppo1/ppo1.py Outdated Show resolved Hide resolved

veds12 reviewed Oct 4, 2020

View reviewed changes

genrl/core/actor_critic.py Show resolved Hide resolved

veds12 requested changes Oct 4, 2020

View reviewed changes

genrl/core/actor_critic.py Show resolved Hide resolved

genrl/core/actor_critic.py Show resolved Hide resolved

genrl/core/actor_critic.py Outdated Show resolved Hide resolved

AdityaKapoor74 and others added 16 commits October 4, 2020 11:39

rollout buffer for MA

2be8df5

Merge branch 'multiagentutils' of https://github.com/AdityaKapoor74/g…

ac9b5a8

…enrl into multiagentutils

Update genrl/utils/utils.py

10282f0

Co-authored-by: Vedant Shah <[email protected]>

Update genrl/agents/deep/ppo1/ppo1.py

79b531b

Co-authored-by: Vedant Shah <[email protected]>

Update genrl/core/actor_critic.py

a3885a0

Co-authored-by: Vedant Shah <[email protected]>

Update genrl/core/actor_critic.py

e3dc677

Co-authored-by: Vedant Shah <[email protected]>

Update genrl/core/actor_critic.py

4c2ad51

Co-authored-by: Vedant Shah <[email protected]>

Update genrl/core/actor_critic.py

43554e4

Co-authored-by: Vedant Shah <[email protected]>

removing SharedAC class

6828e93

removing SharedAC class

eac920c

rectify

194065f

rectify

c0198bc

rectify

fe40835

rectifying

a50204a

removing unecessary code

4a3cd74

removing unecessary code

602a7b5

[WIP] Adding MultiAgent Utilities #323

Are you sure you want to change the base?

[WIP] Adding MultiAgent Utilities #323

Uh oh!

Conversation

AdityaKapoor74 commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

Sharad24 commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

Uh oh!

Uh oh!

Sharad24 Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 3, 2020

Choose a reason for hiding this comment

Uh oh!

Sharad24 left a comment

Choose a reason for hiding this comment

Uh oh!

lgtm-com bot commented Sep 3, 2020

Uh oh!

lgtm-com bot commented Sep 4, 2020

Uh oh!

codecov bot commented Sep 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Sharad24 Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

sampreet-arthi Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Sharad24 Sep 4, 2020

Choose a reason for hiding this comment

Uh oh!

AdityaKapoor74 Sep 5, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Sep 4, 2020 •

edited

Loading